1. Field of the Invention
The present invention relates to a reproducing apparatus, a reproducing system, a reproducing method, and a recording medium.
2. Description of Related Art
Various reproducing apparatus-related technologies for reproducing an image using an interpolation technique for a portion whose image quality has been deteriorated by an abnormal frame of received image data including a plurality of frames during stream delivery have been proposed. Related technology is described below.
Japanese Patent Application Laid-Open (JP-A) No. 62-061485 (patent document 1) discloses technology of performing an interpolation process using the frame.
JP-A No. 62-061485 (patent document 1) discloses an inter-frame prediction decoding apparatus having a structure in which a second frame memory for storing previous frame data from frame data stored in a first frame memory is formed in order to integrate a differential between frames when an inter-frame prediction-coded image signal is decoded, so that the frame data stored in the second frame memory is output when a transmission line error occurs.
When detecting an error, the inter-frame prediction decoding apparatus displays an error-free image on a display unit because data transmitted to the display unit is previous data before the error is generated.
JP-A No. 03-022735 (patent document 2) discloses a technique of switching between an interpolation process and a retransmission process depending on the degree of difficulty in an interpolation process of deteriorated image quality.
An apparatus for transmitting a packet of an image signal according to patent document 2 includes a circuit forming a packet using an interpolation-easy image transmission component even when the image transmission component is lost on a transmission line, a circuit forming a packet using an interpolation-difficult image signal component when the image signal component is lost, and a circuit giving high priority to a packet where precision by which an image is deteriorated is high using a priority display region of the packet when the packet is lost.
In the apparatus for transmitting a packet of an image signal, a packet in a first group is discarded first when a packet is discard on a transmission line. Since the packet in the first group is easy to interpolate and is not important image information, an image is not deteriorated much even when the packet in the first group is discarded.
JP-A No. 2006-174280 (patent document 3) discloses a technique of determining priorities of a plurality of image-deteriorated portions existing inside a frame and performing an interpolation process in order to maintain a real-time characteristic.
An image processing apparatus of patent document 3 includes a defect detector detecting a defective portion from received moving image data, a determining unit determining priority of a defective portion in the case where a plurality of defective portions exist inside one frame of moving image data, and an interpolation processing unit performing an interpolation process on a defective portion according to the priority determined by the determining unit.
An image stretch processor of this image processing apparatus performs a decoding process on moving image stream data received by a communication processor and outputs moving image data. An image defect detector detects a defective portion from moving image data output from the image stretch processor. An interpolation priority determining unit determines priority of interpolation process on a defective portion in the case where a plurality of defective portions exist inside one frame of moving image data. An interpolation processor performs an interpolation process on a defective portion according to priority determined by the interpolation priority determining unit.
Also, there is a technique called forward error correction (FEC), which adds a parity (redundancy data) to predetermined data of a transmission side in order to perform error detection and correction processes by a reception side, and interpolates a loss using a different normal packet which does not need a retransmission process and whose reception has been completed.
Also, JP-A No. 2003-284064 (patent document 4) discloses a technique of displaying an easy-to-see image using an autonomous control at a reception side in the case where an error is detected from image data.
An image receiving apparatus according to patent document 4 includes a receiving unit receiving coded image data, a reproducing unit reproducing received image data, a display unit displaying an image obtained by the reproducing unit, a detecting unit detecting an error of received image data, a measuring unit measuring strength of error durability of received image data, and a switch unit switching an image displayed on the display unit depending on the strength of a measured strength of error durability in the case where an error is detected from received image data.
In this image receiving apparatus, a variable length decoder decodes a frame coding mode, an error detector detects an error of image data during a decoding operation by the variable length decoder, a reception interval measuring unit measures a reception interval of a key frame on the basis of the frame coding mode, and a switch controller switches an image display on a display by controlling connection/cut-off of a switch on the basis of existence of an error, the frame coding mode, and a random access interval.
JP-A No. 2006-319583 (patent document 5) discloses an apparatus for streaming and delivering image and voice data from a transmitter, receiving the image and voice data, and outputting image and voice on the basis of the received image and voice data.
The output apparatus according to patent document 5 receives data transmitted from a different apparatus, performs a loss-compensation process compensating for a loss of the received data when needed, and performs an output process on the basis of the data. The apparatus includes a detecting unit detecting a loss status of received data, a calculating unit calculating a delay time related to data output on the basis of the detected loss status, and a unit delaying the data output depending on the calculated delay time.
The output apparatus receives image and voice data, detects the number of losses that can be compensated for through the loss-compensation process from the received image and voice data, calculates a delay time related to output of the received image and voice data on the basis of a time taken for the loss-compensation process corresponding to the detected number of losses, performs the loss-compensation process compensating for the losses of the image and voice data, delays the image and voice data after the loss-compensation process by the calculated delay time, and then outputs the delayed image and voice data.
However, in a case requiring a high real-time characteristic such as a video conference, it is preferable that an amount of buffering in image data is as small as possible. When retransmission is performed as in patent document 2, an amount of buffering increases and so a real-time characteristic is damaged.
Also, in a case where a retransmission process is impossible as in streaming by multicast, it is impossible to perform a retransmission process described in patent document 2.
Also, in the case where an amount of abnormal frames is large and so an interpolation process is impossible, or when a scene is switched, a sufficient interpolation process cannot be performed using the techniques described in patent document 1 or patent document 3, so that image quality deterioration is caused.
Also, in the FEC, a reception side needs to deal with a parity added by a delivery side. In the case where data to which a parity not defined by the reception side is added is delivered, it is impossible for the reception side to perform an interpolation process.
Also, abnormality of image data described in the specification means that image data itself is lost and does not arrive at the reception side, or a portion of image data is destroyed, not a delay in arrival while image data is received. Also, abnormality of a frame means an incomplete status of a frame after decoding due to abnormality of image data.
Also, prevention of subjective image quality deterioration is not sufficient in the related arts of patent documents 4 and 5.
Here, “subjective” means an image quality appraisal of a frame by a viewer, not an objective image quality appraisal of a frame such as a Peak Signal Noise Ratio (PSNR) value.
Also, “subjective” means “scoring depending on the judgment or psychology of an individual appraisalor”, and “objective” means “routinely scoring without an influence of an appraisalor's subjectivity”. Appraisal of image quality by a viewer's viewing is considered “subjective” because an attitude of accepting image quality depends on the judgment or psychology of an individual even when the same frame is viewed.